Papers with intermediate reasoning steps
BlendSQL: A Scalable Dialect for Unifying Hybrid Question Answering in Relational Algebra (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing hybrid question answering systems use a "prompt-and-pray" paradigm . context size limitations limit ability of many transformer-based LLMs to fit into a given prompt . |
| Approach: | They propose a superset of SQLite to act as a unified dialect for orchestrating reasoning across unstructured and structured data. |
| Outcome: | The proposed framework scales to massive datasets and improves performance while using 35% fewer tokens. |
Do Deep Neural Networks Capture Compositionality in Arithmetic Reasoning? (2023.eacl-main)
Copied to clipboard
Keito Kudo, Yoichi Aoki, Tatsuki Kuribayashi, Ana Brassard, Masashi Yoshikawa, Keisuke Sakaguchi, Kentaro Inui
| Challenge: | Using a pre-trained dataset, we examine how well recent neural models capture compositionality in symbolic reasoning tasks. |
| Approach: | They propose a skill tree on compositionality that defines hierarchical levels of complexity along with three compositionality dimensions: systematicity, productivity, and substitutivity. |
| Outcome: | The proposed model struggled most with systematicity, performing poorly even with relatively simple compositions. |
R3 Prompting: Review, Rephrase and Resolve for Chain-of-Thought Reasoning in Large Language Models under Noisy Context (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have evaluated LLMs under noise-free context but the dilemma for LLM to produce inaccurate results under noisy context has not been fully investigated. |
| Approach: | They propose a new method for CoT reasoning using Chain-of-Thought prompting that interacts with LLMs to perform key sentence extraction, variable declaration and answer prediction. |
| Outcome: | The proposed method outperforms existing CoT prompting methods on five reasoning tasks under noisy context. |
DecoupleSearch: Decouple Planning and Search via Hierarchical Reward Modeling (2025.emnlp-main)
Copied to clipboard
Hao Sun, Zile Qiao, Bo Wang, Guoxin Chen, Yingyan Hou, Yong Jiang, Pengjun Xie, Fei Huang, Yan Zhang
| Challenge: | Retrieval-Augmented Generation (RAG) systems have emerged as a pivotal methodology for enhancing Large Language Models (LLMs). |
| Approach: | They propose a framework that decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. |
| Outcome: | The proposed framework decouples planning and search processes using dual value models, enabling independent optimization of plan reasoning and search grounding. |
Decomposition-Enhanced Training for Post-Hoc Attributions in Language Models (2026.eacl-long)
Copied to clipboard
Sriram Balasubramanian, Samyadeep Basu, Koustava Goswami, Ryan A. Rossi, Varun Manjunatha, Roshan Santhosh, Ruiyi Zhang, Soheil Feizi, Nedim Lipka
| Challenge: | Existing methods for extractive QA struggle in multi-hop, abstractive, and semi-extractive settings. |
| Approach: | They propose a method that prompts models to produce answer decompositions as intermediate reasoning steps. |
| Outcome: | The proposed method outperforms existing methods and matches or exceeds state-of-the-art frontier models. |
Boosting Language Models Reasoning with Chain-of-Knowledge Prompting (2024.acl-long)
Copied to clipboard
| Challenge: | Recent studies have shown that Chain-of-Thought (CoT) prompting can be effective on complex reasoning tasks but generates unfaithful and unfactual reasoning chains. |
| Approach: | They propose a chain-of-knowledge prompting that elicits Large Language Models to generate explicit pieces of knowledge evidence in the form of structure triple. |
| Outcome: | The proposed method improves commonsense, factual, symbolic, and arithmetic reasoning tasks by estimating the reliability of the reasoning chains in terms of factuality and faithfulness. |
Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofs (2025.findings-naacl)
Copied to clipboard
| Challenge: | a new framework for complex reasoning with LLMs is developed to improve reasoning proof accuracy and interpretability. |
| Approach: | They propose to use LLMs to generate search logs that can be interpreted into human-readable reasoning proofs. |
| Outcome: | The proposed framework improves reasoning accuracy but lacks interpretability due to black-box nature of the solvers. |
Distilling Reasoning Capabilities into Smaller Language Models (2023.findings-acl)
Copied to clipboard
| Challenge: | a step-by-step reasoning approach like chain of thought has proved to be effective in eliciting reasoning abilities in large language models. |
| Approach: | They propose a knowledge distillation approach that leverages CoT reasoning capabilities of larger models and distills them into smaller models. |
| Outcome: | The proposed scheme boosts the performance of smaller models over 70% on multiple reasoning datasets. |
PhysPRM: A Generative Process Reward Model with Fine-grained Diagnosis for Physics Problem Solving (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing Large Language Models (LLMs) struggle with physics problem solving due to difficulties in decoding implicit constraints and maintaining physical consistency. |
| Approach: | They propose a Generative PRM that treats evaluation as a generative task . it produces fine-grained diagnoses comprising critiques, final judgments, and specific error types . |
| Outcome: | The proposed model improves performance across seven benchmarks in Best-of-N and critique refinement strategies. |
Efficient Test-Time Scaling of Multi-Step Reasoning by Probing Internal States of Large Language Models (2026.acl-long)
Copied to clipboard
Jingwei Ni, Ekaterina Fadeeva, Tianyi Wu, Mubashara Akhtar, Jiaheng Zhang, Elliott Ash, Markus Leippold, Timothy Baldwin, See-Kiong Ng, Artem Shelmanov, Mrinmaya Sachan
| Challenge: | Existing verification approaches, such as Process Reward Models, are computationally expensive and limited to specific domains. |
| Approach: | They propose a transformer-based probe that uses internal states of frozen LLMs to estimate credibility of reasoning steps during generation. |
| Outcome: | The proposed probes match or exceed PRMs that are up to 810 larger. |
How Many Demonstrations Do You Need for In-context Learning? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are capable of complex reasoning when given a few input-output demos. |
| Approach: | They use fewer input-output demos for each test query to study ICL . they do not observe significant degradation when using only one randomly chosen demo . |
| Outcome: | The proposed model outperforms multi-demo models on the tasks in 2022. |
When LLMs Read Tables Carelessly: Measuring and Reducing Data Referencing Errors (2026.acl-long)
Copied to clipboard
Yuqing Yang, Qi Zhu, Zhen Han, Boran Han, Zhengyuan Shen, Shuai Wang, Vassilis N. Ioannidis, Huzefa Rangwala
| Challenge: | Large language models (LLMs) perform well on table tasks, but they still make data referencing errors (DREs) prior studies have only offered limited, small-scale analyses. |
| Approach: | They propose inference-time strategies and lightweight critics to mitigate data referencing errors. |
| Outcome: | The proposed model achieves an average F1 score of 78.2% in detecting both in-distribution and out-of-difference DREs and assists inference for larger models. |
Table-Critic: A Multi-Agent Framework for Collaborative Criticism and Refinement in Table Reasoning (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to decompose large language models (LLMs) lack effective mechanisms to identify and correct errors in intermediate reasoning steps, leading to cascading error propagation. |
| Approach: | They propose a multi-agent framework that facilitates collaborative criticism and iterative refinement of the reasoning process until convergence to correct solutions. |
| Outcome: | The proposed framework achieves superior accuracy and error correction rates while maintaining computational efficiency and lower solution degradation rate. |
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) are shown to perform better when asked to reason step-by-step before generating a final answer. |
| Approach: | They propose a framework to tailor small-sized LMs to generate correct reasoning steps and robustly reason over these steps. |
| Outcome: | The proposed framework outperforms four competitive baselines and improves the robustness and generalization ability of the reasoning LM, yielding higher performance on out-of-distribution test sets. |
Stepwise Perplexity-Guided Refinement for Efficient Chain-of-Thought Reasoning in Large Language Models (2025.findings-acl)
Copied to clipboard
Yingqian Cui, Pengfei He, Jingying Zeng, Hui Liu, Xianfeng Tang, Zhenwei Dai, Yan Han, Chen Luo, Jing Huang, Zhen Li, Suhang Wang, Yue Xing, Jiliang Tang, Qi He
| Challenge: | Chain-of-Thought (CoT) reasoning has improved the performance of large language models (LLMs) however, the detailed reasoning process in CoT often incurs long generation times and high computational costs due to the inclusion of unnecessary steps. |
| Approach: | They propose a method to identify critical reasoning steps using perplexity as a measure of their importance. |
| Outcome: | The proposed method achieves a better balance between reasoning accuracy and efficiency of CoT. |
TurnBench-MS: A Benchmark for Evaluating Multi-Turn, Multi-Step Reasoning in Large Language Models (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmarks focus on single-turn or single-step tasks, failing to capture iterative reasoning in real-world settings. |
| Approach: | They propose a benchmark that evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |
| Outcome: | The new benchmark evaluates multi-turn, multi-step reasoning through an interactive code-breaking task inspired by the "Turing Machine Board Game" the best model achieves 84% accuracy in Classic mode, but performance drops to 18% in Nightmare mode. |
Reason-KE++: Aligning the Process, Not Just the Outcome, for Faithful LLM Knowledge Editing (2026.findings-acl)
Copied to clipboard
| Challenge: | Current methods for modifying parameters to integrate new knowledge are not accurate enough. |
| Approach: | They propose an SFT+RL framework that instills process-level faithfulness by a stage-aware Reward mechanism and a Stage-assisted Reward Mechanism. |
| Outcome: | The proposed framework instills process-level faithfulness while boosting final accuracy. |
Stance Reasoner: Zero-Shot Stance Detection on Social Media with Explicit Reasoning (2024.lrec-main)
Copied to clipboard
| Challenge: | Stance Reasoner is a model for zero-shot stance detection on social media platforms that can be used to extract opinions from opinionated content. |
| Approach: | They propose a method that leverages explicit reasoning over background knowledge to guide the model’s inference about the document’s stance on a target. |
| Outcome: | The proposed model outperforms the current state-of-the-art models on 3 Twitter datasets, including fully supervised models. |
Program of Thoughts for Financial Reasoning: Leveraging Dynamic In-Context Examples and Generative Retrieval (2025.emnlp-main)
Copied to clipboard
| Challenge: | Numerical reasoning remains a challenging area for large language models (LLMs). |
| Approach: | They propose a two-step framework to enhance LLM's capabilities in financial numerical reasoning by using a generative retriever and context-aware program of thought prompting. |
| Outcome: | The proposed model surpasses previous benchmarks with execution accuracy improvements of 5.98% and 4.05%, respectively. |
Beyond Outcome Verification: Verifiable Process Reward Models for Structured Reasoning (2026.findings-acl)
Copied to clipboard
| Challenge: | Recent work on reinforcement learning with verifiable rewards (RLVR) has shown that large language models can be substantially improved using outcome-level verification signals. |
| Approach: | They propose a framework where intermediate reasoning steps are checked by deterministic, rule-based verifiers. |
| Outcome: | The proposed framework achieves 20% higher F1 than state-of-the-art models and 6.5% higher than verifiable outcome rewards, with substantial gains in evidence grounding and logical coherence. |
SMART: Evaluating LLMs’ Mathematical Reasoning via a Human Cognitive Process-Inspired Benchmark (2026.acl-long)
Copied to clipboard
| Challenge: | Existing evaluation methods focus on the final answer or on the intermediate reasoning steps, overlooking its inherently multi-stage and multi-dimensional nature. |
| Approach: | They propose a benchmark that decomposes mathematical problem-solving into four cognitive dimensions and introduces dimension-specific tasks to measure their cognitive processes. |
| Outcome: | The proposed model decomposes mathematical problem-solving into four cognitive dimensions and introduces dimension-specific tasks to measure their cognitive processes. |
MedCoach: Enhancing Medical Reasoning in LLMs via Knowledge Graph-Augmented Chain-of-Thought Distillation (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for training specialized reasoning models for the medical domain are limited due to the scarcity of high-quality, large-scale Chain-of-Thought (CoT) data. |
| Approach: | They propose a framework that introduces a dedicated coach role to guide the student model through question decomposition. |
| Outcome: | The proposed framework smooths the learning curve in medical reasoning by facilitating domain adaptation before advancing to complex long-chain reasoning. |